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ABSTRACT 


One  of  the  primary  training  tools  available  to  a  Joint  Commander  in  Chief  (CINC)  for 
training  his  staff  on  their  joint  mission  essential  tasks  is  a  command  post  exercise 
supported  by  a  computer  simulation  model.  Computer  Aided  Exercises  (CAXs)  are  an 
essential  part  of  training  a  component  staff,  however  one  weakness  with  these  valuable 
training  tools  lies  in  the  measurement  of  the  level  of  training  received  by  the  players.  In 
most  CAXs  the  players  rapidly  disperse  after  the  exercise,  and  little  quantitative  data  are 
captured  during  the  running  of  the  exercise  that  will  allow  for  quick  post  exercise  analysis. 
This  research  presents  a  methodology  for  evaluating  the  performance  of  joint  intelligence 
tasks  as  set  forth  in  the  Universal  Joint  Task  List.  Instead  of  attempting  to  provide 
individual  measures  for  each  joint  intelligence  task,  the  methodology  presented  focuses  on 
the  analysis  of  significant  events  that  occur  during  an  exercise  and  relating  intelligence 
functions  that  may  have  contributed  to  the  outcome  of  such  events.  Results  of 
experimental  runs  of  the  Joint  Theater  Level  Simulation  are  presented  to  demonstrate  the 
methodology  and  the  subsequent  analysis  process. 


V 


TABLE  OF  CONTENTS 


l.  INTRODUCTION  . 1 

A.  BACKGROUND  .  1 

B.  PROBLEM  STATEMENT  .  2 

C.  THESIS  STRUCTURE  .  3 

II.  OVERVIEW  OF  INTELLIGENCE  . 5 

A.  DEFINITION  OF  INTELLIGENCE .  5 

B.  LEVELS  OF  INTELLIGENCE  .  6 

C.  THE  INTELLIGENCE  CYCLE .  7 

1.  Requirements  .  8 

2.  Information  Collection  .  9 

3.  Information  Processing  .  19 

4.  Dissemination  of  Intelligence  .  11 

D.  KNOWN  PROBLEMS  IN  MEASURING  VALUE  OF  INTELLIGENCE  .  12 

m.  METHODOLOGY  . 

A.  SIGNIFICANT  EVENTS  .  15 

B.  A  THEORETICAL  METHODOLOGY  .  16 

C.  A  PRACTICAL  METHODOLOGY  .  17 

1.  Intelligence  Report  Scoring  .  18 

2.  The  Asset  Needs  Function  (ANF)  .  23 

IV.  APPLICATION  . •  ■  •  •  27 

A.  JOINT  THEATER  LEVEL  SIMULATION  (JTLS)  .  27 

B.  SCENARIO  DESCRIPTION  .  28 

C.  ANALYSIS  OF  TBM  INTELLIGENCE  .  31 

1.  Significant  Events .  ^1 

2.  Report  Scores  .  ^1 

3.  Asset  Needs  Function  .  37 

D.  ANALYSIS  OF  REPUBLICAN  GUARD  INTELLIGENCE  .  40 

1.  Significant  Events .  40 

2.  Report  Scores  .  41 

E.  SUMMARY  .  45 

V.  SUMMARY  AND  RECOMMENDATIONS  . 47 

A.  SUMMARY  .  47 

B.  RECOMMENDATIONS  .  48 

vii 


.  . .  51 


APPENDIX  . 

LIST  OF  REFERENCES  . 

BIBLIOGRAPHY  . 

INITIAL  DISTRIBUTION  LIST 


Via 


Executive  Summary 

One  of  the  primary  training  tools  available  to  a  Joint  Commander  in  Chief  (CENC)  for 
training  his  staff  on  their  joint  mission  essential  tasks  is  a  command  post  exercise 
supported  by  a  computer  simulation  model.  This  is  commonly  referred  to  as  a  Computer 
Aided  Exercise  (CAX).  The  primary  role  of  the  computer  simulation  is  to  present  a 
stochastic  decision  environment  within  which  the  staff  can  be  presented  with  realistic 
results.  Based  on  this  simulated  environment,  staffs  implement  plans,  monitor  the  current 
situation,  and  further  develop  or  alter  those  plans  as  required  by  changing  requirements. 
Computer  Aided  Exercises  are  an  essential  part  of  training  a  component  staff,  however 
one  weakness  with  these  valuable  training  tools  lies  in  the  measurement  of  the  level  of 
training  received  by  the  players.  In  most  CAXs  the  players  rapidly  disperse  after  the 
exercise,  and  little  quantitative  data  are  captured  during  the  running  of  the  exercise  that 
will  allow  for  quick  post  exercise  analysis.  Measurement  of  a  staffs  capability  to  perform 
mission  essential  tasks  is  ultimately  important  for  two  reasons.  First,  it  is  important  to 
insure  that  training  resources  are  being  used  wisely,  and  progress  is  being  realized  in  the 
training  program.  Second,  it  is  important  to  determine  the  tasks  for  which  there  exists  the 
greatest  need  for  further  training.  This  research  is  intended  to  furnish  a  CINC  staff  with  a 
methodology  for  evaluating  the  performance  of  intelligence  functions  in  the  context  of  a 
CAX. 

In  the  development  of  this  methodology,  it  is  insightful  to  regard  the  measure  of  any 
intelligence  process  as  the  answer  to  the  question:  How  well  was  the  information 
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necessary  for  optimizing  the  outcome  of  an  action  provided  in  a  timely,  accurate,  and 
understandable  manner?  An  answer  to  this  question  is  the  goal  for  any  analysis 
methodology.  This  research  provides  two  methodologies  for  measuring  effectiveness  in 
the  performance  of  intelligence  joint  tasks  during  a  CAX.  First,  a  theoretical  methodology 
which  relies  on  a  computer  simulation  model  that  is  currently  not  in  existence  is  discussed. 
Second,  a  more  practical  methodology  which  could  be  used  with  only  slight  modification 
to  some  of  the  computer  simulations  in  use  today  is  presented.  The  practical  methodology 
focuses  on  the  analysis  of  significant  events  that  occur  during  an  exercise  and  relating 
causal  factors  that  may  have  contributed  to  the  outcome  of  such  events. 

This  thesis  also  provides  a  demonstration  of  the  application  of  the  practical 
methodology  using  the  Joint  Theater  Level  Simulation  which  is  currently  in  use  for  the 
training  of  CINC  staffs.  Specifically,  the  demonstration  shows  how  that  with  only  minor 
changes,  a  simulation  model  can  furnish  the  data  necessary  to  provide  greatly  enhanced 
measures  of  how  an  inteUigence  staff  performed  throughout  the  conduct  of  an  exercise. 

The  methodology  presented  does  not  attempt  to  assign  values  to  the  performance  of 
each  individual  joint  intelligence  task  stated  in  the  UJTL,  but  seeks  to  determine  how 
intelligence  functions  contributed  to  the  outcome  of  significant  events  that  occurred  during 
the  exercise.  The  methodology  is  relatively  uncomplicated,  but  retains  the  robustness 
necessary  to  be  applicable  in  many  different  exercise  scenarios.  Since  it  is  uncomplicated, 
it  allows  for  quick  analysis  that  can  be  easily  understood  in  post  exercise  debriefings.  An 
additional  strength  is  that  interaction  with  the  computer  model  is  very  limited  with  most  of 
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the  data  necessary  to  use  the  methodology  readily  available  and  easy  to  output  in  most 
event  step  simulation  models. 
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L  INTRODUCTION 


A.  BACKGROUND 

The  Chairman  Joint  Chiefs  of  Staff  (CJCS)  Memorandum  of  Policy  26  (MOP  26) 
establishes  a  program  for  carrying  out  the  joint  training  responsibilities  of  the  CJCS,  the 
Joint  Commander  in  Chiefs  (CINCs),  and  the  CINC's  component  staffs.  MOP  26 
institutes  a  method  for  identifying  training  requirements  through  the  review  of  the  CINC's 
mission,  and  the  compilation  of  the  CINC's  Joint  Mission  Essential  Task  Lists  (JMETL) 
required  to  accomplish  that  mission.  A  CINC's  JMETL  is  intended  to  provide  the  basis 
for  all  joint  training. 

The  Universal  Joint  Task  List  (UJTL),  a  document  devised  to  support  the  Joint 
Training  Manual  (MCM  71-92),  outlines  a  comprehensive  list  of  joint  essential  tasks.  The 
UJTL  is  intended  to  provide  a  common  language  for  describing  joint  warfighting 
capabilities  throughout  the  entire  range  of  military  operations  to  include  operations  other 
than  war.  Specifically,  tasks  are  defined  as  they  relate  to  the  strategic  (both  national  and 
theater),  operational,  and  tactical  levels  of  war.  Each  joint  task  is  broken  down  into 
supporting  tasks  which  may  be  further  refined  into  enabling  tasks. 

One  of  the  primary  training  tools  available  to  a  CINC  for  training  his  staff  on  their 
joint  mission  essential  tasks  is  a  command  post  exercise  supported  by  a  computer 
simulation  model.  This  is  commonly  referred  to  as  a  Computer  Aided  Exercise  (CAX). 
The  primary  role  of  the  computer  simulation  is  to  present  a  stochastic  decision 
environment  within  which  the  staff  can  be  presented  with  realistic  results.  Based  on  this 
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simulated  environment,  staffs  implement  plans,  monitor  the  current  situation,  and  further 
develop  or  alter  its  plan  as  required  by  changing  requirements.  Computer  Aided 
Exercises  are  an  essential  part  of  training  a  component  staff,  however  one  weakness  with 
these  valuable  training  tools  lies  in  the  measurement  of  the  level  of  training  received  by  the 
players.  In  most  CAXs  the  players  rapidly  disperse  after  the  exercise,  and  little 
quantitative  data  are  captured  during  the  running  of  the  exercise  that  will  allow  for  quick 
post  exercise  analysis.  Measurement  of  a  staffs  capability  to  perform  mission  essential 
tasks  is  ultimately  important  for  two  reasons.  First,  it  is  important  to  insure  that  training 
resources  are  being  used  wisely,  and  progress  is  being  realized  in  the  training  program. 
Second,  it  is  important  to  determine  the  tasks  for  which  there  exists  the  greatest  need  for 
further  training. 

B.  PROBLEM  STATEMENT 

This  research  develops  an  exercise  analysis  methodology  for  evaluating  CINC  staff 
performance  in  the  execution  of  joint  tasks  during  the  conduct  of  a  CAX,  focusing  on 
Strategic  Task  Three,  Develop  Theater  Intelligence,  as  stated  in  the  UJTL.  Specific 
objectives  are : 

1)  Develop  the  analytical  tools  necessary  to  provide  insight  into  the  value  of 
intelligence  information  during  the  conduct  of  a  CAX  designed  to  work  in 
conjunction  with  data  manipulated  by  an  unspecified  computer  simulation. 
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2)  Test  methodology  using  the  Joint  Theater  Level  Simulation  (JTLS). 

Develop  a  standardized  ASCH  file  for  capturing  parameters  and  demonstrate  a 
potential  post-exercise  analysis.  This  objective  entails  a  practical  application  of 
the  methodology  to  an  existing  theater  level  simulation.  Included  in  this  are  the 
alignment  of  the  model's  database  with  required  parameters  necessary  for  utilizing 
the  methodology,  development  of  algorithms  required  in  post  processing,  and 
specification  of  output  format. 

It  is  important  to  emphasize  that  this  research  is  part  of  a  larger  ongoing  research 
project  which  will  attempt  to  provide  an  overall  analysis  methodology  for  all  of  the  joint 
tasks  specified  in  the  UJTL  within  the  context  of  a  CAX.  Concurrent  with  the 
development  of  the  methodology  presented  in  this  paper  was  the  development  of  a 
methodology  for  assessing  the  performance  of  Strategic  Task  Eight,  Provide  Theater 
Sustainment,  presented  in  a  paper  by  Captain  Ray  Combs  U.S.A.  Since  the  performance 
of  one  joint  task  during  a  CAX  often  impacts  the  performance  of  another  joint  task,  it  is 
strongly  recommended  that  the  reader  read  both  papers  in  order  to  gain  insight  into  an 
overall  analysis  methodology  which  will  attempt  to  identify  common  causal  factors  that 
influence  significant  events  that  occur  during  a  CAX. 

C.  THESIS  STRUCTURE 

Chapter  11  provides  a  brief  overview  of  the  Intelligence  Process,  and  some  of  the 
known  problems  in  measuring  the  impact  of  Intelligence  Products.  Chapter  III  will 
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describe  the  proposed  analysis  methodology  used  to  assess  staff  performance.  The 
presented  methodology  focuses  on  the  analysis  of  significant  events  that  occur  during  an 
exercise  and  related  intelligence  functions  that  may  have  contributed  to  the  outcome  of 
such  events.  Chapter  IV  looks  at  applying  the  methodology  to  a  typical  exercise  scenario 
using  the  Joint  Theater  Level  Simulation.  This  chapter  discusses  the  data  manipulation  that 
is  necessary  for  post  exercise  analysis  using  an  existing  computer  simulation.  A  summary 
of  observations  and  possible  inferences  from  the  validation  run  are  included  in  this  chapter. 
Chapter  V  summarizes  the  methodology,  and  provides  recommendations  for  further 
refinements  and  study. 
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n.  OVERVIEW  OF  INTELLIGENCE 


This  chapter  provides  a  general  overview  of  intelligence.  Specifically,  it  attempts  to 
furnish  insight  into  the  underlying  concepts  of  the  structure  of  the  intelligence  tasks  in  the 
UJTL.  In  addition,  this  chapter  addresses  some  of  the  past  problems  that  have 
necessitated  the  need  for  determining  the  value  of  intelligence,  and  the  problems  that  have 
arisen  from  the  attempt  to  assess  that  value. 

A.  DEFINITION  OF  INTELLIGENCE 

Intelligence  is  a  term  for  which  no  clear-cut  definition  exists.  Not  only  does  it  not 
have  a  precise  and  agreed  definition,  but  those  who  attempt  precision  give  it  multiple 
meanings.  The  Department  of  Defense  Dictionary  of  Military  and  Associated  Terms 
(JPl-02)  defines  intelligence  as  "  the  product  resulting  from  the  collection,  processing, 
integration,  analysis,  evaluation,  and  interpretation  of  available  information  concerning 
foreign  countries,  or  areas."  This  definition  is  followed  by  a  listing  of  forty-one 
subcategories  of  definitions  for  intelligence,  thus  demonstrating  the  difSculty  in  containing 
the  definition  of  the  term  intelligence  to  one,  all-encompassing  definition.  It  is  important 
to  distinguish  the  difference  between  information  and  intelligence.  Information  is  the  raw 
material  from  which  inteUigence  is  derived.  To  produce  intelligence,  the  raw  information 
must  be  processed  into  a  product  which  is  accurate  and  relevant.  Although  not  easy  to 
define,  intelligence  does  have  a  clear  purpose  in  that  it  is  intended  to  provide  potentially 
significant  information  to  military  planning  and  operations.  Dr.  R.V.  Jones,  the  head  of 
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the  British  Air  Ministry's  intelligence  section  during  World  War  II,  stated,  "The  ultimate 
objective  of  intelligence  is  to  enable  action  to  be  optimized."  [Ref  1]  This  statement 
highlights  the  overall  goal  for  intelligence,  that  being  to  reduce  uncertainty  and  the 
element  of  risk  in  the  planning  and  execution  of  any  military  operation. 

B.  LEVELS  OF  INTELLIGENCE 

Joint  Military  Intelligence  exists  at  three  levels,  the  highest  level  being  strategic 
intelligence  which  is  required  for  the  formulation  of  strategy,  policy,  and  military  plans  and 
operations  at  the  national  and  theater  level.  The  next  level  is  operational  intelligence 
which  provides  for  conducting  campaigns  and  major  operations  within  a  theater  or  area  of 
operation.  The  lowest  level  is  tactical  inteUigence  which  supports  the  planning  of  battles 
and  engagements,  focusing  at  this  level  on  specific  combat  elements  and  objectives.  These 
three  levels  of  intelligence  compose  the  basic  hierarchy  of  intelligence.  Many  of  the  past 
boundaries  that  existed  between  these  levels  of  intelligence  are  growing  less  clear  with  the 
changes  in  information  management  systems  and  the  rapid  increase  in  technology.  As  an 
example,  satellite  reconnaissance,  once  a  tool  reserved  for  strategic  intelligence,  gradually 
became  an  integral  part  of  operational  intelligence.  Now,  through  such  programs  as 
Tactical  Exploitation  of  National  Capabilities  (TENCAP),  the  development  of  enhanced 
hardware,  software,  and  communications  architecture  will  allow  for  direct  "sensor  to 
shooter"  relays  of  satellite  information  to  the  operator  at  the  tactical  level. 
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C  THE  INTELLIGENCE  CYCLE 


The  most  widely  accepted  model  for  the  intelligence  process  is  referred  to  as  the 
intelligence  cycle  (Figure  1).  The  intelligence  cycle  is  divided  into  four  phases.  The  first 
phase,  requirements,  involves  specifying  the  perceived  need  for  intelligence  and  a  plan  for 
how  to  best  collect  the  information  required  to  satisfy  that  need.  In  the  second  phase, 
collection,  actual  assets  are  tasked  with  the  collection  of  the  information.  The  third 
phase  is  the  processing  of  raw  information  collected  into  an  intelligence  product.  An 
intelligence  product  can  take  many  different  forms,  from  a  formal  National  Intelligence 
Estimate  (NIE)  to  a  simple  verbal  report  given  over  the  telephone.  The  final  phase  is  that 
of  dissemination  of  the  intelligence  product  to  the  user. 


Figure  1.  The  Intelligence  Cycle 
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1.  Requirements 

In  the  cycle  model  of  intelligence,  requirements  are  supposed  to  come  from  the 
decision  makers.  However,  requirements  are  usually  established  in  different  ways,  ranging 
from  explicit  tasking  to  vague  questions  which  are  left  to  an  intelligence  staff  to  interpret 
into  specific  intelligence  requirements.  Intelligence  requirements  may  be  generated  by 
events  that  were  not  anticipated,  but  because  they  have  taken  place,  intelligence  collection 
is  required.  Establishment  of  intelligence  requirements  is  essentially  determining  what  is  to 
be  done,  with  the  necessary  condition  of  who  will  do  it.  Because  of  the  difficulty  in 
explicitly  defining  the  inputs  for  requirement  development,  the  UJTL  addresses  the 
requirements  phase  of  the  intelligence  cycle  in  a  very  general  manner.  The  joint  and 
supporting  tasks  for  the  establishment  of  intelligence  requirements  are  summarized  in 

Figure  2. 


Establishmeni 

Requir 

of  Intelligence 
ements 

ST  3.1  Establish  Intelligence  OP 5.1  Develop' 

Requirements  Intelligen 

ST  3.1.1  Evaluate  Prior 

Intelligence  Requirements 

ST  3.1 .2  Identify.  Prioritize,  and 

Validate  Intelligence 

Requirements 

ST  3.1.3  Develop  Collection  Strategy, 

Collection  Plan,  and  Requests 
for  Information  and  Products 

Operational  TA  4.1  Develop  Tactical  Intelligence 

ce  Requirements  Requirements 

Figure  2.  The  UJTL  joint  and  supporting  tasks  for  intelligence  requirements 
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2.  Information  Collection 


Collection  in  the  intelligence  cycle  refers  to  the  gathering  of  raw  data  through  human 
sources  (special  forces,  espionage,  etc.  ),  technical  means  (photography,  interception  of 
electronic  communications,  etc.),  or  in  any  other  manner  such  as  from  "open  source 
[Ref  2]  The  raw  data  to  be  collected  during  the  collection  phase  are  not  restricted  to  the 
obvious  candidates  such  as  enemy  unit  and  target  location,  but  also  includes  demographics 
of  an  area  of  operation  which  can  play  a  significant  role  in  any  military  operation.  The 
UJTL  separates  the  levels  of  collection  into  the  joint  and  supporting  tasks  depicted  in 
Figure  3  .  Collection  is  the  phase  of  the  intelligence  cycle  that  comes  closest  to  what  is 


most  commonly  considered  intelligence  activity. 


Collection  of  Information 


ST  3.2  Collect  Theater  Strategic 
Information 

ST  3.2.1  Collect  Information  on 
Strategic  Situation, 
Geography, 

Climatology,  and 
Significant  Hazards 

ST  3.2.2  Collect  Information  on 
Theater  Stategic  Targets 


OP  5.2  Collect  Operational 
Information 

OP  5.2.1  Collect  Information 
Enenny  Operational 
Situation  and  Hazards 

OP  5.2.2  Collect  Information  on 
Operational  Targets 


TA  4.2  Collect  Infonnation 

TA  4.2.1  Collect  Information  on 
Situation 

TA  4.2.2  Collect  Target 
Information 


Figure  3.  The  UJTL  joint  and  supporting  tasks  for  intelligence  collection 
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3.  Information  Processing 

Unfortunately,  no  matter  how  good  the  collected  information  is,  it  seldom  provides 
the  useful  information  without  some  analysis  or  processing.  In  the  vast  majority  of  cases 
the  information  collected  is  fragmentary,  ambiguous,  and  susceptible  to  widely  divergent 


interpretations.  Thus,  the  process  of  analyzing  the  available  information  to  make 
judgments  about  capabilities,  intentions,  and  actions  of  another  party  is  a  vital  part  of  the 
intelligence  cycle.  [Ref  2  ]  The  processing  of  raw  information  into  intelligence  takes  a 
wide  variety  of  forms,  from  the  photographic  interpreters  analyzing  imagery  data  to 
analysts  using  super-computers  to  decode  enemy  communications.  Processing  of  raw  data 


into  intelligence  can  take  place  even  on  a  very  low  level  in  the  form  of  a  tactical  operator 


of  a  weapon  system  making  basic  judgments  using  past  experience.  The  UJTL  breaks 


information  processing  into  the  joint  and  supporting  tasks  summarized  in  Figure  4. 


Information  Processing 


ST  3.3  Process  Strategic  Information 

ST  3.3.1  Evaluate  Strategic  Threat 
Information 

ST  3.3.2  Analyze  Theater  Area  of 
Interest 

ST  3.3.3  Integrate  Strategic 
Intelligence 

ST  3.3.4  Develop  Indications  and 
Warnings 

ST  3.3.5  Identify  Operational 
Vulnerabilities 


OP  5.3  Process  Operational  Information  TA  4.3  Process  Information 

^  ^  ,  TA  4.3.1  Evaluate  Threat 

OP  5.3. 1  Evaluate  Operatonal  Information 

Threat  Information 

.  *  K  i;*.  TA  4.3.2  Evaluate  Physical 

OP  5.3.2  Analyze  Area  of  Responsibility  Environment 

/  Joint  Operations  Area 

TA  4.3.3  Evaluate  Social/ 

OP  5.3.3  Integrate  Operational  Political/Economic 

Intelligence  Environment 

OP  5.3.3. 1  Develop  Enemy  Operational  ^  ^  4  integrate  intelligence 

Information 

OP  5.3.3.2  Develop  Operational  Target 
Information 

OP  5.3.3.3  Identify  Enemy  Vulnerabilities 


Figure  4.  The  UJTL  joint  and  supporting  tasks  for  intelligence  processing 
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4.  Dissemination  of  Intelligence 

The  most  commonly  overlooked  phase  of  the  intelligence  cycle  is  that  of 
dissemination.  Dissemination  is  simply  getting  the  right  intelligence  product  to  the  military 
personnel  who  need  it  and  can  use  it.  Intelligence  products  can  be  broken  down  into  three 
categories:  current  intelligence,  basic  descriptive  intelligence,  and  intelligence  estimates. 
Current  intelligence  reports  are  designed  to  provide  intelligence  information  that  may 
affect  operations  in  the  immediate  future.  Included  in  this  category  are  Indications  and 
Warnings,  usually  referred  to  as  I&W,  which  are  designed  to  provide  timely  warning  of 
hostile  action.  I&W  are  based  on  an  analysis  that  when  certain  "indicators"  occur,  such  as 
a  particular  airfield  conducting  air  operations,  there  is  a  likelihood  of  enemy  action. 

Current  intelligence  reports  may  be  issued  daily,  hourly,  or  immediately  upon 
receipt  of  the  information.  Basic  intelligence  is  a  general  type  of  intelligence  product  that 
contains  information  that  is  usually  less  susceptible  to  change,  such  as  an  enemy's  air  order 
of  battle  or  a  hostile  country's  demographics.  Basic  intelligence  reports  are  usually  issued 
on  a  periodic  basis.  Intelligence  estimates  are  the  most  ambitious  type  of  intelligence 
product.  These  estimates  are  supposed  to  take  the  broadest  view  of  the  subject  and 
project  the  current  situation  into  the  future.  A  substantial  effort  is  devoted  to  providing  a 
single  estimate  of  how  a  current  situation  will  evolve.  However,  sometimes  dissenting 
opinions  are  expressed  in  what  has  been  traditionally  called  a  footnote,  even  though  the 
dissent  may  be  included  in  the  text  of  an  estimate.  [Ref.  2]  The  UJTL  breakdown  of  the 
dissemination  of  intelligence  is  summarized  in  Figure  5. 
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Figure  5.  The  UJTL  joint  and  supporting  tasks  for  intelligence  dissemination 


D.  KNOWN  PROBLEMS  IN  MEASURING  VALUE  OF  INTELLIGENCE 

Questions  on  the  value  of  intelligence  are  frequently  asked,  but  are  seldom  answered 
in  specific  terms.  One  such  example  occurs  in  the  weapons  system  procurement  area. 
What  is  the  value  of  a  new  intelligence  terminal  that  will  allow  real  time  overhead  imagery 
to  be  displayed  to  a  tactical  commander?  This  question  can  only  be  answered  by 
answering  another  question.  What  is  the  value  of  real  time  intelligence  in  a  tactical 
situation?  Based  on  this  question  many  large  intelligence  agencies  such  as  the  National 
Security  Agency  (NSA)  have  been  asked  to  justify  their  budgets  measured  in  the  billions 
to  Congress  and  to  other  supervising  bodies.  PRef  3]  In  the  joint  training  environment  the 
question  is  asked,  How  well  are  the  intelligence  staffs  trained  to  provide  for  the  success  of 
a  mission?  Once  again  this  a  question  of  the  value  of  intelligence  produced  by  the  joint 
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intelligence  staff.  There  is  a  general  consensus  that  timely,  accurate,  relevant,  and  well 
presented  intelligence  can  be  seen  as  force  multiplier,  and  the  lack  of  it  can  be  considered 
both  literally  and  figuratively  as  a  force  divider.  Unfortunately,  there  has  been  no  real 
way  to  quantify  the  use  of  intelligence  as  a  force  multiplier.  Like  a  catalyst  in  a  chemical 
reaction,  intelligence,  with  its  intangible  output,  is  known  to  be  a  vital  constituent  of  the 
process  it  influences  so  critically,  but  its  precise  function  is  diflScult  to  isolate  and  measure 
with  accuracy.  [Ref  4] 

Frequently,  when  trying  to  determine  the  value  of  intelligence,  the  issue  of  the  cost 
of  intelligence  arises.  This  issue  is  not  easily  addressed,  since  the  determination  of  the  cost 
of  the  inputs  to  intelligence  products  is  not  always  clear,  neither  is  it  an  easy  problem  to 
attach  a  value  to  the  benefits  fi-om  the  decisions  made  with  aid  of  intelligence,  as  opposed 
to  decisions  made  without  intelligence.  For  example,  say  an  attack  submarine  has  the 
collateral  mission  of  gathering  information  on  enemy  shipping  activities,  and  the 
information  on  enemy  shipping  supplements  information  gained  fi'om  intelligence  agents 
and  assists  a  theater  commander  to  make  a  decision  to  commence  a  blockade.  How 
would  the  cost  of  the  submarine  collecting  information  as  part  of  a  secondary  mission  be 
determined?  What  is  the  value  in  dollars  of  information  that  merely  supplements 
information  used  in  decision  making?  Unfortunately,  cost  of  intelligence  is  an  area  that 
yields  little  insight  into  the  value  of  intelligence. 

A  methodology  for  assisting  the  commander  and  his  staJff  in  assessing  the  value  of 
intelligence  shortfalls  in  its  collection,  and  its  contribution  to  the  total  campaign  during  a 
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CAX  is  described  in  the  next  chapter.  A  demonstration  of  the  application  of  these 
methods  is  presented  in  Chapter  IV. 
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m.  METHODOLOGY 


In  the  development  of  an  exercise  analysis  methodology  for  evaluating  CINC  staff 
performance  in  the  execution  of  joint  intelligence  tasks  during  the  conduct  of  a  CAX,  it  is 
insightful  to  regard  the  measure  of  any  intelligence  process  as  the  answer  to  the  question: 
How  well  was  the  information  necessary  for  optimizing  the  outcome  of  an  action  provided 
in  a  timely,  accurate,  and  understandable  manner?  An  answer  to  this  question  is  the  goal 
for  any  analysis  methodology.  This  chapter  provides  two  methodologies  for  measuring 
effectiveness  in  the  performance  of  intelligence  joint  tasks  during  a  CAX.  First,  a 
theoretical  methodology  which  relies  on  a  computer  simulation  model  that  is  currently  not 
in  existence  is  discussed,  and  second,  a  more  practical  methodology  which  could  be  used 
with  only  sUght  modification  to  some  of  the  current  computer  simulations  in  use  today  is 
presented. 

A.  SIGNIFICANT  EVENTS 

A  common  part  to  both  post  exercise  analysis  methodologies  for  evaluating 
intelligence  staff  performance  presented  is  the  concept  of  a  significant  event.  A 
significant  event  for  this  paper  is  defined  as  any  event  occurring  during  an  exercise  that 
would  be  useful  in  the  post  exercise  reconstruction  for  analysis  of  intelligence  inputs  into 
the  decision  making  process.  Sigmficant  events  can  be  classified  in  terms  of  theater, 
operational,  and  tactical  levels  as  well  as  to  the  degree  of  their  significance,  which  may  be 
somewhat  subjective.  An  example  of  a  significant  event  may  be  that  of  an  Iraqi  invasion 
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of  Kuwait  which  would  also  be  an  event  with  a  very  high  degree  of  overall  significance. 
An  example  of  a  significant  event  at  the  tactical  level  may  be  that  of  an  infantry  platoon 
engaging  in  a  firefight  with  the  enemy  and  sustaining  minor  casualties.  The  degree  of 
significance  of  this  event  would  have  to  be  evaluated  in  relation  to  mission  objectives  and 
expectations. 

The  identification  of  a  significant  event  allows  for  the  potential  determination  of 
how  well  an  intelligence  staff  was  able  to  provide  needed  intelligence  information  by 
analysis  of  intelligence  functions  and  perceptions  that  had  an  impact  on  the  outcome  of 
the  significant  events. 

B.  A  THEORETICAL  METHODOLOGY 

A  problem  with  measuring  an  intelligence  staffs  performance  based  solely  on  the 
outcome  of  significant  events  is  that  it  involves  the  assumption  that,  provided  perfect 
intelligence,  all  decision  making  will  result  in  optimal  event  outcomes.  Unfortunately,  due 
to  multiple  factors  including  the  stochastic  nature  of  combat  engagements,  sub-optimal 
decision  making  can  result  in  sub-optimal  outcomes  to  significant  events,  even  though 
perfect  intelligence  information  has  been  provided. 

In  order  to  minimize  the  impact  of  imperfect  decision  making  and  the  stochastic 
nature  of  most  exercise  simulations,  a  methodology  for  solely  evaluating  an  intelligence 
staffs  ability  to  provide  the  required  intelligence  would  involve  using  a  computer 
simulation  model  where  the  decision  makers  are  part  of  the  simulation,  and  would  make 
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decisions  and  issue  orders  according  to  a  programmed  set  of  decision  rules.  An 
intelligence  staff  would  then  provide  inputs  to  the  model  for  one  run  of  the  simulation,  and 
scenario  outcome  measures  would  be  collected  such  as  the  number  of  casualties,  amount 
of  ordinance  expended,  amount  of  terrain  gained  or  lost,  etc.  .  The  outcome  measures 
would  then  be  compared  to  benchmark  simulation  runs  where  the  model  is  provided 
perfect  information,  or  no  information.  An  intelligence  staffs  performance  would  then  be 
compared  to  the  mean  outcomes  of  the  two  benchmarks,  which  could  be  run  multiple 

times  to  assist  in  the  minimization  of  stochastic  effects. 

This  type  of  methodology  attempts  to  sanitize  the  environment  in  which 
intelligence  products  are  delivered.  A  major  deficiency  in  its  successful  implementation  is 
the  construction  of  decision  rules.  Building  a  set  of  decision  rules  that  would  handle  a 
majority  of  the  situations  would  be  an  enormous  undertaking.  Most  military  leaders 
would  find  it  extremely  difficult  to  provide  the  essential  elements  necessary  for  them  to 
make  decisions,  or  the  weighting  given  to  different,  sometimes  conflicting,  intelligence 

reports. 

C  A  PRACTICAL  METHODOLOGY 

It  is  unlikely  that  any  simulation  in  the  foreseeable  future  will  provide  an 
environment  for  assessing  a  CINC's  intelligence  staff  fi-ee  of  nuisance  variables.  However, 
it  is  possible  to  develop  a  methodology  which  can  work  with  existing  computer  simulation 
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models  that  will  provide  quantifiable  measures  that  can  be  used  to  audit  intelligence 
functions  leading  up  to  significant  events. 

1.  Intelligence  Report  Scoring 

One  insightful  and  manageable  question  involving  the  performance  of  an 
intelligence  staff  is;  How  good  was  the  information  on  the  potentially  hostile  units  in  the 
area  of  interest?  The  concept  of  report  scoring  used  in  this  analysis  attempts  to  answer 
that  question  by  taking  the  approach  that  an  intelligence  report  on  any  Other  Than 
Friendly  Unit  (OTFU)  can  be  decomposed  into  three  essential  report  elements;  location 
of  the  unit,  the  unit  strength,  and  an  estimate  of  the  unit's  course  of  intended  action  .  One 
method  for  measuring  the  accuracy  of  each  report  element  (e.g.,  position  of  an  enemy 
unit)  would  be  to  compare  the  intelligence  reported  position  to  the  actual  or  ground  truth 
position  and  assign  a  score  based  on  the  difference  for  that  particular  type  of  unit.  The 
greater  the  difference  between  the  intelligence  report  element  and  ground  truth  "reality, 
the  lower  the  Report  Score. 

However,  another  important  factor  that  must  be  considered  in  measuring  the 
worth  of  intelligence  is  the  depreciation  of  intelligence  information  value  over  time  .  The 
inherent  value  of  intelligence  may  depreciate  with  time  from  . 

Actual  changes  in  the  situation. 

Possible,  but  unknown  change  -  so  that  the  report  cannot  be  used  with  the  same 

level  of  confidence  as  before. 

It  is  important  to  emphasize  that  intelligence  information  need  not  only  be  accurate,  but 
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that  it  must  also  be  "fresh",  so  that  a  decision  maker  will  remain  confident  in  the  use  of  it. 
The  rate  at  which  the  value  of  an  intelligence  report  element  will  depreciate  is  a  function 
of  the  type  of  OTFU  reported.  For  example,  intelligence  on  the  location  of  an  infantry 
battalion  will  depreciate  faster  than  intelligence  on  the  location  of  an  aircraft  factory. 
[Ref  1] 

Incorporating  the  concept  that  value  of  intelligence  depreciates  with  time,  utility 
weighting  curves  can  be  constructed  for  each  OTFU  type  and  intelligence  report  element 
with  respect  to  time.  Utility  weighting  functions  and  the  resulting  utility  weighting  curves 
express  the  decay  in  the  value  of  an  intelligence  report  on  an  OTFU  as  the  report  is 
allowed  to  age.  Since  the  depreciation  of  intelligence  value  will  vary  with  report  element 
type,  the  attributes  of  these  utiUty  weighting  curves  will  most  likely  be  different  for  each 
intelligence  report  element. 

This  paper  suggests  three  possible  functional  forms,  borrowed  from  the  study  of 

economics,  to  express  the  manner  in  which  intelligence  information  may  be  perceived  to 

depreciate  .  Determining  the  actual  depreciation  functions  for  inteUigence  information  on 

any  particular  unit  for  a  given  scenario  is  challenging,  however  common  properties  to  all 

intelhgence  depreciation  functions  would  be  that  they  are  monotonic  decreasing  and  they 

return  weights  between  zero  and  one,  with  one  representing  the  value  of  perfect 

information.  The  first  depreciation  function  is  simple  linear  depreciation; 

>r(t)  =  -at+l  forO<a<  1, />0,  (1) 

w(t)  =  0  for  at  >  1, 
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where  t  is  time  and  a  is  the  decay  rate.  A  second  depreciation  function  suggested  is  that 
of  "constant  percentage"  depreciation  commonly  used  in  the  calculation  of  depreciation  in 
the  value  of  machinery.  It  could  be  used  for  most  categories  of  intelligence  information 
and  is  expressed  by: 

w(0  =  (l-ay  forO<a<  1,  ^>0  .  (2) 

The  third  function  is  essentially  one  minus  a  logistic  function  and  is  shown  below: 

w(0  =  l-7;^for0<a<l,  P>0,t>0  .  (3) 

Equation  3  has  the  distinguishing  property  of  decreasing  slowly  for  small  values  oft,  and 
then  decreasing  dramatically  faster  as  t  increases,  where  the  shape  parameters,  a  and  P, 
adjust  the  rate  of  decay.  This  functional  form  is  intended  to  model  the  value  of  some  type 
of  information  that  may  retain  much  of  its  value  for  a  certain  period  of  time,  but  after  that 
time  has  passed,  it  quickly  depreciates.  These  three  suggested  depreciation  functions  are 
not  meant  to  capture  every  possible  structure  for  which  information  value  will  depreciate, 
but  merely  to  provide  a  sound  starting  point  in  the  development  of  utility  weighting 
functions  for  the  value  of  intelligence  information.  Figure  6  provides  sample  curves  for 
the  three  functions  given  above  with  specified  decay  and  shape  parameters. 

Currently  most  computer  simulation  models  do  not  provide  for  the  real  world 
problem  of  inaccuracies  in  the  reporting  of  intelligence  information.  Most  simulations 
provide  a  sensor  with  a  baseline  probability  of  detection,  perhaps  adjusted  for  range,  and 
environmental  variables  such  as  jamming,  commonly  multiplied  by  a  target  detection 
multiplier  for  the  target  to  be  observed.  The  simulation  then  performs  a  draw  fi-om  a 
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random  number  generator  to  determine  whether  sensor  A  detects  target  B.  If  a  detection 
is  made,  then  the  simulation  provides  actual  information  on  the  target  without  error. 


DEPRECIATION  IN  VALUE  OF  INTELUGENCE  WITH  TIME 


Figure  6.  Depreciation  of  intelligence  value  with  time. 

There  may  be  some  delay  time  inherent  in  the  simulation  for  the  sensor  information  to  be 
processed,  but  the  information  at  time  of  detection  is  100%  accurate.  Consequently,  the 
only  way  that  intelligence  information  becomes  inaccurate  is  by  the  passage  of  time.  This 
fact  results  in  the  determination  of  the  utility  for  the  OTFU  intelligence  report  elements  of 
location,  estimate  of  course  of  action,  and  strength  simply  as  a  function  of  their  age 
without  including  the  possibility  that  the  information  has  inaccuracies  at  the  time  of 
detection. 

Position,  course  of  intended  action,  and  strength  information  are  the  core  elements 
for  an  intelligence  report  on  an  Other  Than  Friendly  Unit,  and  examples  can  easily  be 
constructed  to  show  cases  where  all  three  elements  would  be  critical  to  good  decision 
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making,  as  well  as  examples  where  knowledge  of  any  one  or  two  elements  would  be 
sufficient.  Since  it  may  prove  difficult  to  determine  the  potential  critical  intelligence  report 
element,  the  Report  Score  will  be  composed  of  a  sum  of  the  values  of  the  three  elements. 
However,  this  methodology  will  still  allow  that  an  OTFU’s  Report  Score  could  be  broken 
down  into  finer  detail  to  provide  the  audit  trail  into  the  values  for  each  individual 
intelligence  report  element,  should  that  be  necessary . 

Typically  a  decision  maker  relies  on  two  important  pieces  of  information  to  make  a 
judgment  on  the  quality  or  value  of  an  intelligence  report.  The  first  is  reliability  of  the 
source  of  the  intelligence.  However,  computer  simulations  generally  do  not  attempt  to 
model  unreliable  information  sources.  The  second  is  age  of  the  intelligence  which  can  be 
modeled  in  most  simulations.  Therefore  the  main  measure  of  how  good  the  intelUgence  is 
on  a  particular  OTFU  will  be  measured  by  the  Report  Score  shown  in  equation  4. 

Report  Score-  (t)  =  —  (4) 

where 

w  -  A  utility  weighting  factor  fi-om  0  to  1  of  the  depreciation  of  intelligence 
data  as  a  function  of  intelligence  report  element  type,  OTFU  type,  and 
age. 

indices; 

i  -  Other  Than  Friendly  Unit  {  1st  Rep  Guard,  2nd  Artillery  Battalion...} 
t  -  time  {in  integer  hours  from  start  of  C AX  t  =  0, 1 ,2. . . . } 
j(t)  -  age  of  last  intelligence  update 

1  -  intelligence  report  element  type  { location,  estimate  of  CO  A,  strength  } 
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The  Report  Score  can  provide  a  measure  of  how  effective  a  CINC's  intelUgence 
staff  was  at  providing  valuable  information  on  OTFUs  with  only  limited  assumptions  as  to 
the  structure  of  the  decay  of  the  value  of  the  information  as  it  is  allowed  to  age. 
Combined  with  the  identification  of  significant  events  occurring  during  an  exercise,  and 
the  corresponding  significant  OTFUs,  the  Report  Score  will  furnish  some  insight  into  the 
ability  of  an  intelligence  staff  to  furnish  "fresh"  information. 

2.  The  Asset  Needs  Function  (ANF) 

One  of  the  most  demanding  functions  performed  by  an  intelligence  staff  is  the 
tasking  of  intelligence  assets.  Poor  intelligence  collection  asset  allocation  was  specifically 
noted  as  a  deficiency  for  Operation  Desert  Shield/Storm.  [Ref  5]  In  large,  complex 
exercises  the  assessment  of  the  surpluses  and  shortages  of  collection  assets  that  occur  can 
provide  insight  into  how  well  an  intelligence  staff  made  use  of  those  collection  assets  at  its 
disposal.  To  adequately  address  the  question  of  why  intelligence  information  on  an  enemy 
unit  was  deficient,  it  is  necessary  to  determine  whether  collection  assets  were  not 
available,  or  were  merely  imder  utilized. 

An  important  aspect  of  the  problem  of  collection  asset  allocation  is  the 
determination  of  the  potential  need  for  any  particular  collection  asset  or  type  of  collection 
asset  at  any  given  time.  The  fi'amework  for  measuring  an  intelligence  staffs  ability  to 
adequately  provide  collection  asset  coverage  within  a  theater  of  operation  will  be  centered 
on  maintaining  a  record  of  each  collection  asset's  availability,  and  the  potential  need  for 
that  asset  at  any  time  during  an  exercise.  An  intelligence  collection  asset  is  considered  to 
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be  available  if  it  is  determined  that  it  could  be  tasked  by  the  intelligence  staff  during  a 
specified  time  window  to  conduct  a  collection  mission.  Determination  of  the  potential 
need  for  any  collection  asset  at  a  specific  time  is  slightly  more  involved.  Potential  need  is 
established  by  whether  there  exists  a  significant  Other  Than  Friendly  Unit  or  units  that 
have  a  suflBciently  low  Report  Score,  and  whether  there  exists  a  collection  asset  that  has  a 
sufficiently  high  probability  of  detection  for  any  of  those  significant  OTFUs.  The 
purpose  of  the  Asset  Needs  Function  is  to  show  the  existence  of  a  perceived  need  for  a 
particular  collection  asset  to  provide  information  on  a  particular  OTFU  at  a  specific  time 
in  the  exercise.  The  Asset  Needs  Function  can  be  written  in  the  form: 

ANFy,  (t)  =  (l-(Report  ScorCi  (t))  x  pdy,  (t)  x  IMFj  (t)  x  SRF^  x  TRFy,  (5) 
where  the  indices  are  : 

i  -  Other  Than  Friendly  Unit  {1st  Rep  Guard,  2nd  Artillery  Battalion,...} 
k  -  intelligence  collection  asset  {  JSTARS,  TR-1,  HUMINT  teams,...} 
t  -  time  (in  hours  from  start  of  CAX  t  =  0,1,2,....}, 
and  the  component  variables  are  defined  as  follows  ; 

pdij,(t)  -  the  probability  of  detecting  OTFU,  i,  at  time  t,  given  that  OTFU,  i  is 

within  sensor  range  of  collection  asset,  k . 

IMFi(t)  -  an  importance  factor  assigned  for  the  degree  of  significance  of 
OTFU,  i ,  at  time,  t. 

SRFfc  -  a  sensor  range  factor  to  adjust  for  the  difference  in 

volume  of  search  area  covered  by  the  different  sensors  earned  by  the 
collection  asset,  k. 
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TRF.^  -  a  target  range  factor  to  compensate  for  the  range  of  the  target  from  the 
staging  point  of  the  collection  asset.  Essentially,  this  implies  that  targets 
at  the  extreme  limits  of  a  collection  asset's  ability  to  search  may  be 
harder  to  detect. 

For  the  ANF  to  return  a  high  value,  the  Report  Score  on  OTFU,  i ,  must  be  low,  and  the 
probability  of  detection  by  collection  asset,  k  ,must  be  sufficiently  high.  In  summary,  the 
Asset  Needs  Function  is  intended  to  express  the  potential  of  a  collection  asset  to  improve 
the  Report  Score  of  an  Other  Than  Friendly  Unit. 

Once  the  ANF  values  have  been  determined  for  each  collection  asset,  it  is  possible 
to  classify  each  by  whether  there  exists  a  potential  need  for  that  asset,  and  whether  it  is 
available  for  tasking.  Each  collection  asset  at  time,  t,  would  be  in  one  of  four  disjoint 
states; 

State  One  -  there  existed  a  need  for  the  collection  asset  manifested  by  a  high 

ANF  for  an  OTFU  of  significance,  and  the  asset  was  available  to  be 
tasked. 

State  Two  -  there  existed  a  need  for  the  collection  asset  manifested  by  a  high 
ANF  Report  Score  for  an  OTFU  of  significance,  and  the  asset  was 
not  available  to  be  tasked. 

State  Three  -  there  did  not  exist  a  need  for  the  collection  asset,  and  the  asset 
was  available  to  be  tasked. 

State  Four  -  there  did  not  exist  a  need  for  the  collection  asset,  and  the  asset 
was  not  available  to  be  tasked. 

Of  course,  some  threshold  values  will  have  to  be  established  for  an  ANF  to  be  considered 
high. 

By  identifying  significant  events  that  occur  during  a  Computer  Aided  Exercise,  and 
subsequently  determining  which  units  were  significant  to  the  outcomes  of  those  events,  it 
is  possible  in  post  exercise  analysis  to  use  the  Report  Scores  to  gain  insight  into  the  ability 
of  the  intelligence  staff  to  provide  the  highest  value  intelligence  possible.  In  addition,  by 
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analysis  of  collection  asset  availability,  and  the  ANFs  for  collection  assets,  it  is  possible  to 
show  potential  shortages  and  surplus  in  the  allocation  of  assets  to  collect  information. 
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IV.  APPLICATION 


This  chapter  demonstrates  an  application  of  the  practical  methodology  for  evaluating 
the  performance  of  an  intelligence  staff  described  in  Chapter  III,  using  a  computer  model 
currently  in  use  for  the  training  of  CINC  staffs.  Specifically,  it  shows  how  that  with  only 
minor  changes,  a  simulation  model  can  furnish  the  data  necessary  to  provide  greatly 
enhanced  measures  of  how  an  intelligence  staff  performed  throughout  the  conduct  of  an 
exercise.  It  is  important  to  emphasize  that  this  experiment  was  not  intended  to 
demonstrate  tactics  or  to  evaluate  the  performance  of  the  computer  model. 

A.  JOINT  THEATER  LEVEL  SIMULATION  (JTLS) 

JTLS  is  an  interactive,  multi-sided,  joint  (air,  land,  naval,  and  special  operations)  and 
combined  (coalition  warfare)  constructive  simulation  model.  This  computer-based 
wargaming  system  uses  inherent  functions  -  sea,  air,  land,  special  operations,  intelligence, 
and  logistics  -  to  model  conflict  (pre-combat,  combat  operations,  and  post  combat)  at  the 
operational  level  of  war  with  tactical  fidelity.  The  Joint  Warfighting  Center  is  the  joint 
sponsor  of  JTLS,  and  has  used  JTLS  (release  1.85)  as  the  exercise  driver  for  the  combined 
exercise  Keen  Edge  94  in  support  of  U.S.  Forces  and  Japan's  Joint  Self-Defense  Forces, 
and  for  a  combined  U.S.  Thailand  exercise ,  Cobra  Gold  95,  among  others. 

A  new  open  system  version  of  JTLS  (release  2.0)  which  provides  a  much  improved 
user  interface  was  utilized  to  support  this  study.  The  scenario  for  the  study  was  adapted 
from  an  existing  Desert  Storm  scenario  with  orders  being  scripted  to  limit  operator 
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interaction  with  the  model  during  the  simulation  runs.  The  ability  to  enter  all  necessary 
orders  before  the  execution  of  a  simulation  run  allowed  for  multiple  repetitions  of  the 
same  scenario  to  be  run  with  different  random  number  seeds.  In  this  experiment  the  unit 
orders  for  seven  simulation  days  were  entered  prior  to  the  first  simulation  run,  and  three 
repetitions  of  the  same  scenario  were  run  to  determine  the  difference  in  the  output  data 
resulting  from  purely  stochastic  events. 

Routines  for  capturing  the  parameters  required  to  implement  the  methodology  were 
developed  by  Rolands  and  Associates  Inc.,  the  primary  contractor  for  JTLS.  The 
calculation  of  Report  Scores  during  each  simulation  run  required  a  file  to  be  maintained 
that  contained  the  time  of  each  detection,  and  the  time  that  an  intelligence  update  was 
issued  for  each  Other  Than  Friendly  Unit  (OTFU).  Although  not  required  for  calculating 
Report  Scores,  it  was  beneficial  to  output  to  the  file  the  mission  name  that  made  the 
detection  of  the  OTFU  to  assist  in  post  exercise  analysis.  In  addition,  a  separate  file  was 
maintained  which  contained  the  number  of  each  intelligence  collection  asset  available  by 
squadron  and  the  time  whenever  a  change  in  availability  status  occurred.  Both  output  files 
were  in  a  standard  ASCII  format  that  was  easily  read  into  a  commercial  spreadsheet 
package.  Sample  spreedsheets  of  both  output  files  are  contained  in  the  appendix. 

B.  SCENARIO  DESCRIPTION 

The  simulated  intelligence  staff  for  this  sample  exercise  had  the  assigned  mission  of 
designing  and  executing  a  collection  plan  for  the  entire  theater  of  operations.  Two  areas 
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of  primary  interest  were  the  tracking  of  Iraqi  mobile  tactical  ballistic  missiles  (TBMs)  and 
the  monitoring  of  the  five  Iraqi  Republican  Guard  divisions  in  Iraq  at  the  beginning  of  the 
exercise.  Collection  assets  were  limited  to  six  aircraft  types  in  eight  different  squadrons 
operating  out  of  four  bases  in  theater.  Table  1  provides  a  summary  of  the  detection 
capabilities  of  the  six  aircraft  types  and  the  JTLS  squadron  designations  for  the  squadrons 
that  operated  them  during  the  exercise.  The  aircraft  types  for  the  experiment  were  created 
by  altering  the  sensor  performance  parameters  of  existing  aircraft  in  the  JTLS  database. 
The  specific  purpose  of  changing  the  sensor  parameters  was  to  assist  in  exercising  the 
analysis  methodology,  and  should  not  be  considered  representative  of  the  performance 
characteristics  of  actual  aircraft.  Baseline  Probability  of  Detection  (BPD)  is  the  baseline 
probability  in  JTLS  that  a  sensor  wiU  detect  a  target  or  unit.  In  JTLS  when  a  target  or 
unit  is  within  range  of  a  sensor,  the  sensors  BPD  is  multiplied  by  the  targets  Target 
Detection  Multiplier  (TDM)  and  any  relevant  environmental  factors  such  as  jamming  or 
bad  weather,  and  the  product  is  the  probabiUty  of  detection  of  the  target  by  the  sensor. 
The  Range  Factor  (RF)  was  determined  by  the  range  of  the  sensor  which  for  airborne 
sensors  is  measured  in  hexes.  Therefore,  for  this  analysis,  if  a  sensor  has  a  range  of  two 
hexes,  it  would  have  a  RF  of  two.  Mean  Fusion  Time  (MFT),  which  is  measured  in  hours, 
is  the  mean  of  an  exponential  distribution  of  how  long  it  takes  from  sensor  detection  to 
dehvery  of  the  information  to  the  player.  If  an  aircraft  has  a  MFT  of  zero,  it  is  considered 
to  be  a  real  time  sensor. 
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Aircraft 

Type 

Baseline 
Probability  of 
Detection(BPD) 

Mean  Fusion 
Time  (MFT)  in 
hours 

Range  Factor 
(RF) 

Operating 

Squadrons 

Total  Number 
of  Aircraft 

Aircraft  A 

0.5 

0 

2 

94FS 

24 

Aircraft:  B 

0.2 

0 

1 

389FS 

24 

Aircraft  C 

0.4 

1 

1 

152ANG 

12 

Aircraft  D 

0.5 

1 

2 

VF-84 

VF-154 

20 

Aircraft  E 

0.2 

1 

1 

VFA-87 

VFA-195 

24 

Aircraft  F 

0.5 

0.5 

1 

3UAVCO 

12 

Table  1.  Summary  of  Reconnaissance  Aircraft  Types 


The  collection  plan  developed  called  for  the  theater  to  be  divided  into  three  major 
sections.  The  first  section  is  the  portion  of  Iraq  where  the  TBM  batteries  would  be  forced 
to  operate  in  order  to  launch  attacks.  The  second  section  is  a  300  km  wide  corridor  from 
Baghdad  to  the  Kuwait  border.  The  third  section  is  Kuwait,  and  a  zone  100  km  into  Iraq 
along  the  Kuwaiti  border.  These  three  major  sections  were  patrolled  by  the  use  of  eight 
different  air  reconnaissance  routes  for  each  section. 

An  Air  Tasking  Order  (ATO)  for  each  day  would  task  squadrons  to  fly  missions  at 
designated  times  along  specified  routes.  Utilizing  the  JTLS  squadron  designations, 
squadrons  94FS,  389FS,  and  152ANG  were  assigned  to  the  TBM  section  patrols,  while 
VF-154  patrolled  the  corridor  to  Baghdad,  and  the  remaining  squadrons  patrolled  Kuwait 
and  the  Iraqi  border.  It  is  important  to  emphasize  that  although  the  JTLS  squadron 
designations  are  used  throughout  this  application  no  conclusions  should  be  drawn  as  to  the 
capabilities  or  characteristics  of  actual  squadrons.  All  missions  were  composed  of  two 
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aircraft.  A  total  of  144  missions  (288  sorties)  were  scheduled  to  be  flown  on  each  of  the 
seven  days  of  the  exercise. 

C.  ANALYSIS  OF  TBM  INTELLIGENCE 

The  intelligence  staff  for  the  simulated  exercise  allocated  three  squadrons  for  a  total 
of  60  aircraft  for  the  task  of  reporting  the  eight  different  mobile  TBM  batteries  which 
were  expected  to  be  operating  in  a  200  km  x  200  km  area.  The  collection  plan  was  that 
each  squadron  would  be  responsible  for  flying  missions  composed  of  two  aircraft  every 
four  hours  along  predetermined  search  routes  each  day.  The  goal  of  this  collection  plan 
was  to  provide  the  most  up-to-date  information  possible  on  each  of  the  eight  mobile  TBM 
batteries. 

1.  Significant  Events 

In  an  actual  exercise,  critical  events  would  be  determined  for  post  exercise  analysis. 
Examples  of  possible  critical  events  in  this  exercise  would  be  a  massive  launch  of  TBMs 
on  a  specific  day,  or  the  random  launches  of  TBMs  over  the  entire  period  of  the  exercise. 
To  facilitate  the  demonstration  of  the  analysis  methodology  for  the  case  of  the  mobile 
TBMs,  the  missile  launches  will  be  considered  to  be  randomly  distributed  throughout  the 
exercise,  thus  the  entire  time  span  of  the  exercise  is  considered  to  be  one  significant  event. 

2.  Report  Scores 

In  the  calculation  of  Report  Scores,  the  first  question  that  must  be  considered  is: 
What  is  the  expected  decay  rate  and  functional  form  for  representing  the  depreciation  of 
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intelligence  for  the  Other  Than  Friendly  Unit  (OTFU)  reported?  For  this  case  of  the 
reporting  of  TBM  batteries,  only  the  location  information  will  be  considered,  since 
strength  and  course  of  intended  action  information  would  not  be  expected  to  change 
significantly  in  the  limited  exercise  scenario  considered.  A  base  depreciation  function  for 
TBM  batteries  over  the  course  of  the  exercise  is  the  constant  percentage  depreciation 
function  shown  in  equation  2  with  a  decay  rate  of  0.2  per  hour  (20%  per  hour)  for  utility 
weights  used  in  the  Report  Score  for  specific  TBM  batteries.  Two  alternative 
depreciation  functions  are  presented  to  show  the  sensitivity  of  the  Report  Score  to 
changes  in  the  functional  form  and  the  decay  rates  of  the  depreciation  function.  The 
alternative  depreciation  functions  are  a  constant  percentage  depreciation  with  a  decay  rate 
of  0.1  per  hour  (10%  per  hour),  and  the  logistic  functional  form  of  equation  3  with  shape 
parameters  a=0.8  and  P=100.  Figure  7  shows  the  three  depreciation  functions  described 


above. 


Figure  7.  Utility  weight  functions  used  in  analyzing  TBM  intelligence  Report  Scores 
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Utilizing  the  base  constant  percentage  depreciation  of  20%  per  hour  Figures  8  and  9 
show  the  Report  Scores  for  two  of  the  eight  TBM  batteries  for  the  seven  simulation  days 
of  repetition  one.  Each  of  the  "peaks”  in  the  plots  corresponds  to  an  intelligence  update 
resulting  from  a  sensor  detection  of  the  TBM  battery,  and  the  "valleys"  correspond  to  the 
depreciation  in  value  of  the  updates  as  they  are  allowed  to  age.  Note  that  some  of  the 
"peaks"  in  the  Report  Score  do  not  start  at  one,  because  the  fusion  time  inherent  in  some 
sensors  results  in  a  delay  in  the  time  it  takes  for  an  intelligence  update  to  be  delivered  from 
the  time  of  detection. 
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1 


REPORT  SCORE  FOR  TBM  BATTERY  G 


If  the  random  launches  of  TBM  over  the  entire  seven  day  exercise  were  regarded  as 


being  significant  events,  then  it  would  be  of  interest  as  to  how  well  the  TBM  batteries 
were  being  detected  over  that  total  period.  Figures  8  and  9  indicate  where  there  existed 
long  periods  of  low  Report  Scores.  One  area  of  particular  interest  is  the  period  around 
hour  80  (  the  fourth  day)  in  which  Report  Scores  for  both  TBM  batteries  were  basically 
zero  for  almost  an  entire  day.  In  fact,  the  Report  Scores  for  all  eight  TBM  batteries 
indicate  low  Report  Scores  around  the  fourth  day.  The  low  Report  Scores  on  the  fourth 
day  of  the  simulation  would  indicate  that  closer  analysis  should  be  done  into  the  potential 
causal  factors  for  the  reduced  detections,  and  how  they  relate  to  the  overall  campaign 


plan.  Some  causal  factors  for  day  four  were  that  Aircraft  A,  the  best  sensor  for  detecting 
TBMs,  was  tasked  with  patrolling  less  prolific  search  routes,  and  was  suffering  from  lower 
aircraft  availability  due  to  maintenance.  An  important  note  is  that  in  this  limited 
demonstration  exercise  no  attrition  was  played  for  either  collection  assets  or  TBM 


34 


batteries.  If  attrition  had  been  played,  then  the  campaign  plan  may  have  called  for 
prosecution  of  the  mobile  TBM  batteries  early,  and  the  collection  plan  may  have  been 
adequate.  However,  if  the  early  attacks  were  unsuccessful  then  the  window  of  low  Report 
Scores  on  day  four  may  indeed  denote  a  problem. 

Figure  10  shows  a  graphical  representation  of  how  the  Report  Score  for  TBM 
battery  A  would  appear  using  the  three  different  depreciation  functions  given  in  Figure  7. 
Although  three  different  depreciation  functions  were  used.  Figure  10  shows  the  same 
pattern  of  low  Report  Scores  on  day  four.  The  differences  between  the  Report  Scores 
utililizing  slightly  different  depreciation  functions  can  become  more  pronounced  by  the 
selection  of  a  threshold  value,  perhaps  representing  a  minimum  acceptable  level,  and 
calculating  the  amount  of  time  spend  above  or  below  this  threshold. 
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Report  Score  for  IBM  Battery  A  Using  Constant  Percentage  Depreciation  (0.8) 


20  40  60  80  100  120  140  160  180 

TIME  IN  HOURS 


Report  Score  for  TBM  Battery  A  Using  Logistic  Depreciation 


Figure  10.  Report  Scores  for  TBM  Battery  A  using  three  different  depreciation  functions 
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3.  Asset  Needs  Function 


From  Table  1,  it  can  be  seen  that  of  the  three  types  of  aircraft  tasked  with  locating 
the  mobile  TBM  batteries,  Aircraft  Type  A  had  the  greatest  potential  for  detection  with 
BPD  of  0.5  and  SRF  of  two.  Consequently,  for  demonstration  of  the  Asset  Needs 
Function  (ANF)  Aircraft  Type  A  will  be  the  sample  asset.  The  ANF  defined  in  Chapter  III 
will  be  simplified  for  this  presentation  by  holding  all  factors  constant  with  the  exception  of 
Report  Scores  for  each  OTFU.  A  value  of  0.8  was  assigned  to  the  Target  Detection 
Multiplier  (TDM)  for  the  mobile  TBM  batteries,  hence  the  probability  of  detection  for 
Aircraft  Type  A  was  0.4,  which  is  BPD  x  TDM.  The  assumptions  for  the  ANF 

components  for  Aircraft  Type  A  and  mobile  TBM  batteries  used  are  summarized  below: 

pd  =  0.4, 

IMF=1, 

SRF=2, 

TRF=1. 

Thus  the  ANF  for  k  =  Aircraft  Type  A,  with  respect  to  i  =  mobile  TBM  batteries,  at 
time,  t ,  is  : 

ANF-y.  (t)  =  (1 -Report  Score^  (t))  x  0.4  x  2  . 

In  this  simplified  case  the  ANF  is  almost  the  compliment  of  the  Report  Score,  so  a  "pe^" 
in  the  Report  Score  will  result  in  a  "valley"  in  the  ANF,  and  vice  versa.  Note  that  this 
should  not  always  be  assumed  to  be  the  case,  since  a  decrease  in  Report  Score  may  be  the 
result  of  a  factor  that  contributes  to  a  decrease  in  the  ANF  as  well.  For  example,  heavy 
jamming  or  extremely  bad  weather  may  render  a  particular  sensor  completely  ineffective 
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against  an  OTFU,  so  both  the  Report  Score  and  the  ANF  for  that  OTFU  will  decrease,  as 
the  collection  asset's  detection  capabilities  are  degraded. 


One  potential  use  of  the  ANF  is  to  group  the  exercise  into  time  intervals  (e.g., 
days),  and  compute  daily  average  ANFs  for  a  particular  asset.  Table  2  shows  such  an 
example  for  Aircraft  A,  and  the  mobile  TBM  batteries. 


Average 

ANF 

TBM 
Battery  A 

Average 

ANF 

TBM 
Battery  B 

Average 

ANF 

TBM 
Battery  C 

Average 

ANF 

TBM 
Battery  D 

Average 

ANF 

TBM 
Battery  E 

Average 

ANF 

TBM 
Battery  F 

Average 

ANF 

TBM 
Battery  G 

Average 

ANF 

TBM 
Battery  H 

Day  1 

0.5 

0.32 

0.51 

0.4 

0.47 

0.55 

0.45 

0.65 

Day  2 

0.28 

0.44 

0.26 

0.34 

0.43 

0.16 

0.27 

0.35 

Day  3 

0.43 

0.28 

0.39 

0.42 

0.39 

0.33 

0.46 

0.46 

Day  4 

0.79 

0.74 

0.7 

0.74 

0.75 

0.76 

0.68 

0.79 

Day  5 

0.5 

0.34 

0.26 

0.4 

0.38 

0.35 

0.37 

0.25 

Day  6 

0.45 

0.43 

0.25 

0.4 

0.42 

0.63 

0.26 

0.47 

pay  7 

0.78 

0.78 

0.74 

0.74 

0.74 

0.8 

0.66 

0.79 

Table  2.  Average  Daily  ANF  Values  for  Aircraft  A  and  TBM  Batteries. 


By  using  the  daily  averages  in  Table  2  the  ANF  for  a  particular  collection  asset  and  a 
set  of  OTFUs,  it  is  possible  to  determine  which  time  periods  exhibited  the  greatest  need 
for  the  collection  asset,  and  to  prioritize  the  requirement  for  the  collection  assets  to  track 
specific  OTFUs  .  Note  the  relatively  small  amount  of  variability  between  the  ANFs  for  the 
TBM  batteries  on  any  given  day.  Also  note  that  days  four  and  seven  had  the  largest  ANF 
in  comparison  to  the  other  days  of  the  scenario.  Using  daily  average  ANFs  has  the 
additional  advantage  of  allowing  for  the  comparison  of  whether  the  collection  asset  was 
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available  or  not  on  any  particular  day.  One  way  to  determine  asset  availability  is  by 
calculating  the  average  number  of  aircraft  available  at  any  time  throughout  the  day.  By 
comparing  average  collection  asset  availability  with  average  ANFs,  it  is  possible  to  show 
potential  surpluses  and  shortages  of  the  asset.  For  example,  examination  of  Table  2 
reveals  high  average  ANF's  for  Aircraft  A  and  all  TBM  batteries  on  days  four  and  seven, 
but  comparison  to  Table  3  shows  on  average  only  two  of  Aircraft  A  were  available  in 
theater.  With  an  average  of  only  two  Aircraft  A  available  it  is  doubtfiil  that  any  additional 
sorties  could  have  been  generated  that  potentially  would  have  improved  the  reporting  of 
the  mobile  TBM  batteries.  On  day  seven,  however,  there  existed  an  average  of  9.9 
Aircraft  A  available  throughout  the  day,  which  may  have  allowed  for  the  tasking  of 
additional  sorties  which  may  have  resulted  in  an  improvement  in  the  reporting  of  TBM 
batteries. 


Average  #  of 
Aircraft  A 

Available 

pay  1 

14.8 

Day  2 

8.1 

Day  3 

3.9 

Day  4 

2 

Day  5 

12.6 

Day  6 

11.8 

Day  7 

9.9 

Table  3 .  Average  Number  of  Aircraft  A  Available  on  Each  Day  of  the  Exercise 
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Once  preliminary  analyses  of  the  ANFs  for  critical  OTFUs  and  collection  assets  have 
revealed  time  intervals  in  which  there  existed  high  need  for  a  collection  asset,  as  on  days 
four  and  seven,  investigation  into  possible  causal  factors  can  be  pursued.  Investigation 
into  causal  factors  for  the  case  of  mobile  TBM  batteries  should  start  by  analyzing  air 
taslfing  in  the  daily  ATOs,  then  searching  for  common  features  of  days  with  low  asset 
need  and  days  with  high  asset  need  for  particular  collection  assets.  In  addition,  asset 
availability  should  be  analyzed  looking  for  trends  that  may  have  resulted  in  periods  of  high 
ANFs. 

D.  ANALYSIS  OF  REPUBLICAN  GUARD  INTELLIGENCE 

The  intelligence  staff  for  the  simulated  exercise  was  tasked  with  the  surveillance  of 
the  largest  perceived  enemy  threat  in  the  theater,  the  five  Iraqi  Republican  Guard  Divisions 
held  in  reserve  at  the  start  of  the  scenario.  The  Republican  Guard  Divisions  were 
expected  to  be  highly  mobile  and  capable  of  quick  entry  into  any  engagement.  Intelligence 
was  limited  in  that  only  Aircraft  D  had  the  performance  characteristics  necessary  to 
operate  in  the  region  where  the  Republican  Guard  Divisions  were  expected  to  be  located. 
This  limited  the  efforts  to  track  the  five  divisions  to  the  two  aircraft  squadrons  equipped 
with  Aircraft  D,  a  total  of  20  aircraft. 

1.  Significant  Events 

On  the  first  day  of  the  scenario,  Iraqi  forces  in  Kuwait  attack  into  Saudi  Arabia. 
This  invasion  is  considered  to  be  a  significant  event  in  the  exercise.  A  key  area  of  interest 
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for  post  exercise  analysis  of  the  performance  of  intelligence  functions  is  how  well  the  Iraqi 
Republican  Guard  Divisions  were  tracked  on  the  first  day  of  the  scenario.  This  part  of  the 
application  of  the  methodology  is  intended  to  illustrate  the  potential  analysis  of  a  one  day 
time  period  out  of  a  larger  exercise  time  fi-ame. 

2.  Report  Scores 

Once  again,  the  first  question  in  the  calculation  of  the  Report  Score  for  an  OTFU 
that  must  be  considered  is:  What  is  the  expected  decay  rate  and  functional  form  for 
representing  the  depreciation  of  intelligence  information  for  that  OTFU?  For  this  case  of 
the  five  Republican  Guard  Divisions,  a  base  depreciation  function  is  the  constant 
percentage  depreciation  function  expressed  in  equation  (2)  and  illustrated  in  Figure  7  with 
a  decay  rate  of  0.2  per  hour  (20%).  Also,  as  in  the  TBM  example,  only  the  location 
information  will  be  considered.  Figures  11  through  15  show  graphical  representations  of 
the  value  of  intelligence  for  the  five  Republican  Guard  Divisions  throughout  the  first  day. 
It  is  important  to  point  out  that  Aircraft  D,  which  was  the  only  aircraft  type  tasked  with 
the  reporting  of  the  Republican  Guard  units,  was  equipped  with  a  non-real  time  sensor 
with  a  MFT  of  1.0  hours.  The  MFT  results  in  a  delay  from  the  time  of  detection  of  a  unit 
by  the  sensor  to  the  time  of  a  report  being  issued.  Thus,  when  the  report  is  received  the 
intelligence  information  has  already  been  allowed  to  age.  Because  of  this  delay  from 
detection  to  reporting,  and  the  decay  rate  of  0.2  per  hour,  the  average  Report  Score  for  a 
Republican  Guard  Division  at  the  time  of  a  report  is  0.8,  instead  of  the  1.0  that  would  be 
expected  of  a  real  time  sensor.  Also,  since  MFT  is  the  mean  of  an  exponential  distribution 


41 


sampled  by  the  computer  model  to  determine  the  actual  fusion  time  from  detection  to 
reporting,  it  is  not  uncommon  to  have  reports  being  issued  in  excess  of  three  hours  after 
detection  occurred.  This  is  evident  in  Figure  1 1  where  the  first  report  issued  at  the  four 
hour  point  is  from  a  detection  almost  three  hours  prior. 

Analyses  of  the  Report  Scores  for  the  five  Republican  Guard  divisions  during  the 
first  day  of  the  scenario  reveal  some  interesting  points.  First,  Figure  12  shows  that  no 
reports  were  issued  on  the  10th  Armor  Division  until  the  15  hour  mark.  Also  note  that 
both  the  52nd  Armor  and  the  Baghdad  Divisions  suffered  from  low  Report  Scores  during 
the  first  15  hours  of  the  exercise.  Second,  the  Report  Scores  for  all  five  Republican  Guard 
Divisions  demonstrate  the  disadvantage  of  a  non-real  time  sensor  compared  to  what  would 
have  been  expected  of  a  real  time  sensor  in  that  the  time  spent  between  detection  and 
reporting  results  in  a  much  lower  average  Report  Score  over  the  time  period. 


Figure  1 1.  Day  one  Report  Score  for  Iraqi  RepubUcan  Guard  Hammurabi  Division 
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Rwort  Score  eg'  Report  Score 

dcToooooo  C 


Day  One  Report  Score  for  10th  Armor  Division 


re  12.  Day  one  Report  Score  for  Iraqi  Republican  Guard  10th  Armor  Division 


Day  One  Report  Score  for  Baghdad  Division 


Figure  13.  Day  one  Report  Score  for  Iraqi  Republican  Guard  Baghdad  Division 
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23 


Report  Score 


E.  SUMMARY 


This  chapter  has  demonstrated  through  the  use  of  small,  illustrative  examples  how  the 
methodology  can  be  applied  to  a  proven  simulation  model.  The  methodology  is  intended 
to  provide  for  post  exercise  analysis  of  intelligence  functions  through  the  identification  of 
significant  events  and  the  use  of  measures  which  attempt  to  capture  the  value  of 
intelligence  for  key  units.  In  addition,  the  methodology  shows  the  potential  for  improving 
the  inteUigence  picture  of  the  battlespace  by  looking  at  the  availability  and  need  for 
collection  assets.  This  experiment,  conducted  with  JTLS,  has  established  how,  with  only 
limited  interaction  with  the  model,  all  of  the  information  necessary  to  implement  the 
methodology  can  easily  be  output. 
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V.  SUMMARY  AND  RECOMMENDATIONS 


A.  SUMMARY 

This  research  has  provided  a  post  exercise  analysis  methodology  for  evaluating  the 
performance  of  intelligence  functions  during  a  computer  aided  exercise.  The  methodology 
presented  does  not  attempt  to  assign  values  to  the  performance  of  each  individual  joint 
intelligence  task  stated  in  the  UJTL,  but  seeks  to  determine  how  intelligence  functions 
contributed  to  the  outcome  of  significant  events  that  occurred  during  the  exercise.  The 
first  step  in  the  implementation  of  the  methodology  is  to  determine  the  significant  events. 
Next,  determine  how  well  the  intelligence  staff  was  able  to  provide  detection  and 
subsequent  updates  of  the  Other  Than  Friendly  Units  of  interest  throughout  the  exercise 
by  analysis  of  unit  report  scores.  Lastly,  by  analysis  of  Asset  Need  Functions  for  critical 
units  during  the  exercise  and  collection  asset  availability,  determine  where  there  may  have 
existed  potential  shortages,  and  surpluses,  in  allocation  of  assets. 

One  strength  of  the  methodology  is  that  it  is  relatively  uncomplicated,  but  retains  the 
robustness  necessary  to  be  applicable  in  many  different  exercise  scenarios.  Since  it  is 
uncomplicated,  it  allows  for  quick  analysis  that  can  be  easily  understood  in  post  exercise 
debriefings.  An  additional  strength  is  that  interaction  with  the  computer  model  is  very 
limited  with  most  of  the  data  necessary  to  use  the  methodology  readily  available  and  easy 
to  output  in  most  event  step  simulation  models. 
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B.  RECOMMENDATIONS 


A  fundamental  step  in  the  implementation  of  the  methodology  is  the  determination  of 
the  depreciation  functions  for  intelligence  information  described  in  Chapter  III.  Through 
the  use  of  surveys,  simulation,  and  historical  analysis  it  is  possible  to  develop  a  library  of 
depreciation  functions  which  correspond  to  the  intelligence  report  elements  for  specific 
unit  types  (e.g.,  position  information  on  mobile  TBM  batteries).  Such  a  library  would  be 
essential  in  the  development  of  a  packaged  post  exercise  processor  capable  of  rapidly 
processing  the  large  quantities  of  data  generated  in  an  actual  exercise. 

Theater  level  models  by  design  usually  lack  tactical  fidelity,  and  JTLS  is  no  exception. 
For  this  reason,  the  JTLS  database  was  modified  to  provide  collection  assets  with 
reasonable  capabilities  of  what  might  be  expected  for  the  aircraft  types  described  in 
Chapter  IV.  To  truly  evaluate  intelligence  fimctions,  especially  the  use  of  specific 
coUection  assets,  the  JTLS  database  would  need  to  be  modified  to  present  a  more  accurate 
representation  of  actual  collection  assets  and  sensor  loads  capabilities. 

JTLS  also  does  not  attempt  to  model  uncertainty  or  inaccuracy  in  the  reporting  of 
target  information  upon  detection.  As  discussed  in  Chapter  III,  as  with  many  theater 
simulation  models,  the  information  on  the  target  is  perfect  at  the  time  of  detection. 
Although  modeling  imperfect  detection  information  is  a  challenging  task,  one  candidate 
method  would  be  to  perturb  the  detection  information  depending  on  the  detecting  sensor 
and  the  target,  and  then  assign  confidence  levels  with  the  report  of  the  information  to  the 
player. 
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The  development  of  a  post  exercise  processor  to  automate  calculations  necessary  to 
furnish  Asset  Needs  Functions  and  Report  Scores  would  offer  the  ability  to  quickly  sort 
information  necessary  for  analysis.  Chapter  IV  of  this  research  demonstrated  how  a 
methodology  for  evaluating  intelligence  functions  can  be  implemented  in  a  small 
experimental  exercise.  Real  world  exercises  commonly  consist  of  hundreds  of  players  and 
can  run  over  a  period  of  weeks.  To  pro\dde  timely  post  exercise  analysis  of  the  potentially 
vast  quantity  of  data,  it  is  essential  that  an  automated  post  exercise  processor  be 
developed.  The  post  exercise  processor  should  be  contain  a  graphical  interface  capable  of 
generating  a  host  of  tables  and  charts  to  facilitate  timely  analysis  and  debriefing  of  results. 
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APPENDIX 


A.  JTLS  OUTPUT  FILE  FOR  UNIT  DETECTIONS 

The  report  below  is  a  sample  of  the  JTLS  output  data  imported  directly  into 
Microsoft  Excel  for  the  detection  of  mobile  TBM  batteries  in  the  experimental  exercise 
discussed  in  Chapter  IV. 


Time 

update 

was 

issued  in 
hours 
from  start 
ofCAX 

JTLS 

unit/target 

name 

JTLS 
sensor 
name  that 
made  the 
detection 

Probability 

of 

detection 
for  the 

sensor 

Mission 

name 

Time  of 
detection 
in  hours 
from  start 
OfCAX 

26.36667 

C.BY.SCUD 

SURF3 

0.32 

sbx14c 

20.45 

28.23333 

H.BY.SCUD 

SURF2 

0.4 

sbx515 

28.23333 

28.26667 

F.BY.SCUD 

SURF2 

0.4 

sbx115 

28.23333 

28.33333 

F.BY.SCUD 

SURF2 

0.4 

sbx115 

28.283:^ 

28.36667 

A.BY.SCUD 

SURF2 

0.4 

sbx515 

28.35 

28.58333 

F.BY.SCUD 

SURF2 

0.4 

sbx115 

28.23333 

28.65 

G.BY.SCUb 

SURF2 

0.4 

sbx315 

28.65 

H.BY.SCUD 

SURF2 

0.4 

sbx515 

28.58333 

28.71667 

A.BY.SCUD 

SURF2 

0.4 

sbx515 

28.7 

28.8 

C.BY.SCUD^ 

SURF2 

0.4 

sdr715 

28.78333 

29.26667 

C.BY.SCUD 

SURF2 

0.4 

sdr715 

28.78333 

31.88333 

A.BY.SCUb 

SURF2 

0.4 

sbx315  ^ 

31.88333 

32.11667 

ABY.SCUb 

SURF2 

0.4 

sbx315 

31.88333 

32.36667 

H.BY.SCUD 

SURF1 

0.16 

sbx216 

32.46667 

A.BY.SCUD 

SURF2 

0.4 

sbx315 

32.43333 

32.46667 

F.BY.SCUD 

SURF2 

0.4 

sbx315 

32.43333 

32.61667 

G.BY.SCUb 

SURF2 

0.4 

sbx115 

32.58333 

32.68333 

C.BY.SCUD 

SURF2 

0.4 

sdr715 

32.63333 

33 

A.BY.SCUD 

SURF2 

0.4 

sbx515 

32.98333 

36.11667 

H.BY.SCUD 

SURF2 

0.4 

sbx315 

36.11667 

36.13333 

H.BY.SCUD 

SURF2 

0.4 

sbx515 

36.11667 

36.15 

H.BY.SCUD 

SURF2 

0.4 

sbx115 

36.08333 

36.2 

H.BY.SCUD 

SURF2 

0.4 

sbx315 

36.11667 

36.3 

A.BY.SCUD 

SURF2 

0.4 

sbx115 

36.28333 

36.51667 

C.BY.SCUD 

SURF2 

0.4 

sbx515 

36.58333 

D.BY.SCUb 

SURF2 

0.4 

sdr715 

36.583331 
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B.  JTLS  OUTPUT  FILE  FOR  COLLECTION  ASSET  AVAILABILITY 

The  report  below  is  a  sample  of  the  JTLS  output  data  imported  directly  into 
Microsoft  Excel  for  the  availability  of  collection  assets  in  the  experimental  exercise 


discussed  in  Chapter  IV. 


JTLS 

squadron 

name 

Time  in 
hours 
from  start 
ofCAX 

Number  of 

aircraft 

available 

3.UAV.CO 

24.15 

6 

VF-154 

24.16667 

4 

152RG.ANG 

24.41667 

7 

VFA-87 

24.58333 

8 

152RG.ANG 

24.68333 

8 

152RG.ANG 

24.93333 

9 

3.UAV.CO 

25.36667 

8 

VFA-195 

25.41667 

6 

3.UAV.CO 

25.46667 

6 

VFA-195 

25.46667 

8 

VFA-195 

25.5 

9 

VF-84 

25.58333 

7 

VFA-87 

25.6 

10 

152RG.ANG 

25.65 

11 

152RG.ANG 

25.75 

12 

VF-84 

25.85 

9 

VF-84 

25.95I 

7 

VF-84 

25.95 

5 

VFA-87 

26 

8 

VFA-195 

26 

7 

VFA-195 

26 

5 

VF-154 

26 

2 

VF-154 

26 

0 

VF-154 

26.26667 

2 

94FS 

26.36667 

10 

94FS 

26.36667 

8 

94FS 

26.36667 

6 

VF-84 

26.53333 

6 

VF-154 

26.56667 

4 

VFA-195 

26.63333 

6 

VF-154 

26.63333 

6 

94FS 

26.71667 

4 
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